Infinite tri-symmetric group, 
multiplication of double cosets, 
and checker topological field theories 

Yury NERETIIsQ 

We consider a product of three copies of infinite symmetric group and its repre- 
sentations spherical with respect to the diagonal subgroup. We show that such rep- 
resentations generate functors from a certain category of simplicial two-dimensional 
surfaces to the category of Hilbert spaces and bounded linear operators. 

1 Introduction 

1.1. Infinite symmetric group. Denote by Soo the group of all permutations 
of the set N = {1, 2, 3, . . . }. By Soo(a) we denote the stabilizer of points 1, 2, 
. . ., a e N. We also assume Soo(0) = Soo- 

A topology on the group S'oo is defined by the condition: subgroups S'oo (a) 
are open and cosets 5 Soo (a) C Soo form a basis of topology. The group Soo is 
a totally disconnected topological group. 

Classification of irreducible unitary representations of Soo is rather simple, 
all representations are induced from trivial representations of Young subgroups 
S mi x • • • x S mk x S 00 _j2m t see Lieberman [3J, see also Olshanski [TT], and 
exposition in [5], VIII, 1-2. 

By S^ n C Soo w e denote the group of finite permutations of N, this group 
is an inductive limit of finite symmetric groups, 

of in _ I ,oo c 
°oo — u n=l D n- 

The group S^ n is a wild (not type I, see e.g., pQ) discrete group, and de- 
scription of its representations seems to be a non-reasonable problem. 

1.2. n-symmetric groups. Consider the product 

G^ ^ '.— Soo X • • ■ X Soo 

of n copies of Soo- We write elements g G G^ as collections g = (gi, . . . , g n ), 
where gj G Soo- Denote by K the diagonal subgroup in G^ n \ i.e., the subgroup 
consisting of collections (g, g, . . . , <?), where g € Soo- 

We define the n-symmetric group G = GW as the sub group of GM consisting 
of collections (gi, . . . , g n ) such that 

g.g- 1 E Sf? for all i, j < n. 

In other words, G^ consists of all collections 

(5/11,5/12, . . . ,gh n ) such that 5 g Soo, hj G S£ n . 

1 Supported by grant FWF, project P19064 



Denote by K{a) the image of Soo (a) under the diagonal embedding K 
(gM. We define the topology on from the condition: the subgroups K(a) 
are open. In other words, the topology of K ~ is the same as above, the 
quotient-space G^l/if is countable and equipped with the discrete topology. 

1.3. Bisymmetric group. The existing representation theory of infinite 
symmetric groups is mainly the representation theory of the bisymmetric group 
G [21 , see Thoma [14], Vershik, Kerov [15], Olshanski [13], Okounkov [9], Kerov, 
Olshanski, Vershik [2]. The situation was explained by Olshanski in [13] . We 
refer the reader to this paper. 

The group G^ is outside Olshanski's approach to infinite-dimensional groups, 
based on imitation of symmetric pairs, see Olshanski [T^] , [T3] , and also [5] . How- 
ever, CI"' is a (G, K )-pair in the sense of [5], VIII. 5. My another standpoint 
was a strange Nessonov's theorem, [7], [8] about spherical functions on cer- 
tain infinite-dimensional groups with respect to certain small (non-symmetric) 
subgroups. 

A discussion of bisymmetric groups is contained in Addendum to this paper. 

1.4. Content of the paper. The tri-symmetric group. In Sections 
2-4 we discuss the tri-symmetric group G := G^l. 

Section 2. We describe of double cosetfl The space H\G/K K (a)\G/K(J3). 
More precisely, we identify a double coset with a triangulated compact two- 
dimensional surface equipped with certain additional data ('checker-boards'). 

Next, we construct a natural multiplication (a, b) i— > a o b of double cosets, 

K(a)\G/K(j3) x K(p)\G/K(j) -» K(a)\G/K(/3) 

for each a, (3, 7 <E Z+. Thus we get a category § = S^, whose objects are 0,1, 
2, . . . and morphisms (3 — > a are double cosets K(a) \ G/K(f3). 

Multiplications of double cosets is a usual phenomenon for infinite-dimensional 
groups (Ismagilov-Olshanski multiplicativity, see numerous constructions in 5 , 
see also some additional examples in Russian translation of [5], Addendum E, 
and in [3]). 

In this case, we get a construction similar to so-called 'topological field the- 
ories n. 

Section 3. Let p be a unitary representation of G in a Hilbert space H. 
Denote by H(a) the space of K (a)-fixed vectors. Then each a £ K a \ G/Kp 
determines a well-defined operator p(a) : H(f3) — * H(a) and 

p{a)p(b) = p(a o b) for o G K(a) \ G/K(j3), b e K{0) \ G/K (7), 

2 Let G be a group, H, K its subgroups. A double coset is a subset in G of the form 
HgK, where g £ G. The space H \G/K is the set, whose points are double cosets. Denote 
by T(H \ G/K) the space of functions on H \ G/K If a group G is finite, then there is a 
well-defined convolution J r (H 1 \ G/H 2 ) x T{U 2 \ G/H 3 ) -> ^(#1 \ G/H 3 ). But a product of 
individual double cosets is not well-defined. 

3 Topological field theories are imitations of conformal field theories on topological level. 
Usually, this term is used for functors from categories of bordisms to the category of lin- 
ear spaces and linear operators. Bordisms of simplicial complexes were considered, e.g., by 
Natanzon [4]. 
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i.e., we get a representation of the category S. 

Section 4. We construct a family of representations of G, write expressions 
for their spherical functions and for matrix elements of representations of the 
category §. 

1.5. n-symmetric groups. General n-symmetric groups are similar to 
the tri-symmetric group. Generally, checker-boards are tiled by n-gons. Also, 
there is another (noncquivalcnt) construction of a two-dimensional polygonal 
complex from a double coset. We discuss this in Section 5. 

1.6. Abstract theorems. Sections 1-5 form the main part of the paper. In 
supplementary Section 6, we obtain some abstract theorems about unitary rep- 
resentations of n-symmetric groups: uniqueness of a if-spherical vector, rough 
separation of the set of representations, etc. 

Acknowledgements. I am grateful to G. I. Olshanski and N. I. Nessonov 
for discussions of this topic. 

2 Multiplication of checker- boards 

Here we consider the tri-symmetric group G := G' 3 '. 

2.1. Construction of simplicial complexes. We have 3 copies of the set 
N, say red, yellow, and blue. An element of G is a triple of permutations of N, 
denote it by 

(fjred> fjyellow: Qblue) • 

We draw a collection of disjoint oriented black triangles Aj, where j ranges 
in N, and paint their sides in red, yellow, and blue anti- clockwise. We also draw 
a collection of oriented white triangles Bj and paint sides in red, yellow, and 
blue clockwise. 

Next, we glue a simplicial complex from these triangles. If g re d sends i to 
j, then we identify the red side of the black triangle A4 with the red side of 
the white triangle Bj. We repeat the same operation for g ye iiow and guue- In 
this way, we get a disjoint countable union of 2-dimensional compact closed 
triangulated surfaces. 

All components except finite number consist of two triangles (black and 
white, glued along the corresponding sides). We call such components chebu- 
rek^, see Figure [2j 

Remark. Elements of the subgroup K C G correspond to countable collec- 
tions of disjoint chebureks. 

2.2. Checker-boards. Now we define a checker-board (see Figure [IJ as a 
countable disjoint union of triangulated closed surfaces equipped with following 
data: 

a) Triangles are painted in black and white, neighbors of black triangles are 
white and vise versa. 

4 A street fast-food in some countries. 
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Notation for colors, 
red: — — 
yellow: ^— ^— 
blue: 



Figure 1: A checker-board and 3 substitutions 



\ 



Figure 2: Gluing of a cheburek 



b) Edges are painted in red, yellow, and blue. The boundary of any black 
(respectively white) triangles is composed of red, yellow, and blue edges situated 
anti-clockwise (resp., clockwise). 

c) Black (respectively, white) triangles are enumerated by natural numbers. 

d) Almost all components are chebureks, i.e., unions of two triangles. 

Proposition 2.1 There is a canonical one-to-one correspondence between G 
and the set of all checker-boards. 

One-side construction was given above. Conversely, take a checker-board. 
Let a red (yellow, blue) edge be common for i-th black triangle and j-th white 
triangle. Then g re d sends i to j, see Figure [TJ 

2.3. Vertices of checker-boards. There are 3 types of vertices according 
colors of edges at a vertex: red-blue, red-yellow, yellow-blue. 
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Proposition 2.2 The set of red-blue vertices is in a natural one-to-one corre- 
spondence with the set of independent cycles of g r ed9uu e ■ Moreover, the number 
of triangles meeting at a vertex is the double length of the corresponding cycle. 

This is obvious, see Figure [1] 

2.4. Double cosets and checker- boards. We say that an (a, j3)-board 
is a compact (generally, disconnected) oriented triangulated surface equipped 
with the following data: 

SS 

a) triangles are painted in white and black, edges are painted in red, yellow, 
blue as above, 

b) we assign labels 1, 2, . . . , /3 to some black triangles and 1, 2, . . . , a to 
some white triangles. 

SS 

c) There is no chebureks without labels. 

Proposition 2.3 There is a canonical one-to-one correspondence between the 
set K(a) \ G/K(f3) and the set of all (a, /?)- boards. 

Indeed, take a double coset, choose its representative S G and compose the 
checker-board for g. Next, we remove numbers > (3 from black triangles and 
numbers > a from white triangles. Now almost all components are chebureks 
without numbers (we call them empty chebureks), and we remove them. 

Thus we identify double cosets and checker-boards. 

2.5. Multiplication of checker- boards. Consider two checker-boards, 

aeK(a)\G/K(p), b e K{fi) \ G/K (7). 

For each j = 1, 2, . . . , (3 we cut off the black triangle of a with label j and 
the white triangle of b with label j and identify edges of the triangles according 
colours and orientations. We obtain an (a, 7)-board. 
Denote this operation by nob. 

Thus we get a category, whose objects are 0, 1, 2, ... , and morphisms /3 — ► a 
are (a, /3)-boards or, equivalently, double cosets. 

The real reason for this definition is Theorem 13.11 about representations. 
Now we wish to interpret the definition in terms of the group G. 

SS 

2.6. Weak zeros. Let us write elements of K ~ Soo as 0-1 matrices. We 
say that a sequence hj in K{a) is a weak zero if hj converges element-wise to 

the (a + 00) x (a + oo)-matrix 

2.7. Another definition of the multiplication of double cosets. 
Theorem 2.4 Given 

a e K{a) \ G/K((3), b € K{fi) \ G/K (7), 
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Figure 3: Gluing of checker-boards. On the resulting surface, the interior part 
of the fat triangle is removed. 



G 



choose p € a, q € fa, and a weak zero hj in K([3). Then 



Cj := K(a)phjCiK(j) 

equals c = a o fa for sufficiently large j . 

PROOF. First, consider two elements p, q E G and the corresponding 
checker-boards *p, 0. Then the product pq in the group G corresponds to 
the following operation: 

— we remove black triangles from ^ and white triangles from 0; 

— we glue perimeters of corresponding 'holes' in <p and (according colors 
and orientations of edges). 

Second, let h £ K C G. Then the product p/iq corresponds to the following 
operation: for each j we glue j-th white triangle of £2 with h-j-th black triangle 
of «p. 

Let pet Let draw the corresponding checker-board p. Let us remove all 
chebureks, whose black side has label > (3 and white side has label > a, denote 
by V the corresponding reduced checker board. We say that the right (resp. 
left) (a, (3) -support of p is the set of all labels on black (resp. white) triangles 
of V. 

Equivalently, j is not in the right support iff 

j > (3 and p red j = p ye ii ow j = Pbiue > P- 

Now, let hj be a weak zero in K([3). For sufficiently large j, the permutation 
hj G K((3) sends the left support of q to the complement of the right support 
of p. 

Now let p, q e G, let Q be the corresponding checker-boards. We 
evaluate phjQ by the rule described above. Then all black triangles of with 
labels > (3 are glued with chebureks, whose white labels are big. A gluing of a 
cheburek does not change a surface. Big label will be forgotten. 

The same holds for white triangles of *}3 with labels > (3. Thus, we get the 
operation of gluing of (a, /3)-board and (/3, 7)-board described above. □ 

2.8. Involution in the category §. The map p i— > p _1 induces a map 
K{a)\G/K{(3) - K(J3)\G/K(a). 

For an (a, /3)-board a we make a (f3, a)-board o n by the follow rule: 

a) we change 'black' and 'white'; 

b) we change the orientation of the surface. 
Evidently, 

(aob) D = fa D oa n . 



7 



3 Representations. Abstract theorem 

3.1. Operators p(a). Let p be a unitary representation of G — G^ in a 
Hilbert space H . Denote by H (a) the space of K (a)-fixed vectors. Denote by 
P(a) the operator of orthogonal projection to the space H(a). 
Fix a, (3. For p £ G, consider the operator 

H(/3) -> H(a) 

given by 

p(g) := P(a)p(p) = P(a)p(p)P(/3). 
For ri G K (a), r^ G -KX/3) we have 

p(ngr 2 ) =p{g). (3.1) 
Thus p is a function on double cosets K{a) \ G/K({3). 

Theorem 3.1 For any unitary representation p of G for each a, 13, 7 G Z + , 
for each 

aeK{a)\G/K(0), b G K(fl\G/Kfr), (3.2) 

we /iave 

p(a)p(b) = p(ao b). 

Thus we get a representation of the category S of checker-boards. Precisely, 
for each a we assign the Hilbert space H (a) and for each (a, /3)-board a we 
assign the operator p{a) : H((3) — > H{a). 

Evidently p is a ^-representation, i.e., 

P(a n )=p(a)*. 

3.2. Lemma on representations of S^. For representations of S^, 
see [3], [ID], [13] j 0- For proof of the following Proposition 13.21 see, e.g., [5], 
Corollary 8.1.5. 

For a unitary representation of Sao in a Hilbert space if, we define subspaces 
H(a) C H as above. 

Proposition 3.2 Let hj be a weak zer^ in Soda). Then p(hj) converges to 
P(a) in the weak operator topology. 

3.3. Proof of Theorem 13.11 Let a, b be double cosets as above (|3.2[) , let 

p, q G G be their representatives. Let uj be a weak zero in K((3). Then 

p(o)p(b) = P(a)p(p)P(/3)p( q )P( 1 ) = 

= lim P(a)p(p)p(uj)p(q)P(<y) = P{a) lim p{puj(\)P{"f) 

j — >oo J — >oo 

where lim denotes the weak limit. 

For sufficiently large j, a product pu^q is contained in the double coset ao b. 
Therefore, we get p(a o b). □ 

5 See Subsection 12.61 
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4 Representations. Constructions 

In this section we construct some representations of the tri-symmetric group 
and of the category S. 

4.1. Tensor product construction. Here we imitate the construction of 
[15] , [13]. Let U, V, W be (finite dimensional or infinite dimensional) Hilbert 
spaces. Let Uj, Vj, Wk be their orthonormal bases. Consider the tensor product 

H := U <g> V <g> W. 

Fix a unit vector h £ H, 

h=Y2 h vk u, (g> Vj (g) w k eU<g>V <g>W, \\h\\ = 1. 

Consider the tensor product 

H := (H, ft) <g> (if, h) ® ■ ■ ■ = [U ®V ®W, h) ® {U ®V ®W, h) ® ... 

of infinite number of copies of Hilbert spaces H with the the distinguished vector 
ft (see the von Neumann definition of infinite tensor products, [16]L Denote 

£ := ft <g> ft® ••• 6 H. 

Now we wish to define a certain representation of the group G in the 
space H. Let p = (pi,P2,P3) S G. Then pi acts as a permutation of factors U, 
P2 as permutation of factors V, and p$ as a permutation of factors W. 

Let formulate this more carefully. The operators of permutations are well 
defined 

- for pi, p 2 , P3 6 S^ n ; 

- for elements (r, r, r) £ K, they act by simultaneous permutations of 
factors (U ® V ® W) and preserve the distinguished vector £. 

These operators generate a unitary representation of G. 

Remark. The group Soo x x 5oo does not act in this tensor product. □ 

Remark. Let A, B, C be unitary operators in U, V, W respectively. Then 

v h — v (A®B®C)h- 

However, we can not simplify an expression for h by such transformations. □ 

4.2. Spherical functions. Our representations are JC-spherical in the 
following sense: 

Proposition 4.1 The vector £ is a unique K-fixed vector in H. 

Proof. Indeed, choose an orthonormal basis e±, e^, ■ ■ ■ 6 U ® V ® W with 
e\ — h. Expand a K -fixed vector rj G H in Fourier series , 

V = J! C M 2 - e h ® ej 2 (8> • • • 

il,i2,---; where ijy — 1 f° r large ./V 
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Coefficients c must be invariant with respect to all permutations of indices. 
Let some ik ^ 1 and Ci x i 2 ... = e ^ 0. Then there is a countable number 
of pairwise different permutations of the sequence i\, ...and therefore a 
countable number of Fourier coefficients = e. Therefore 1 1 77 1 1 = 00 . □ 

Next, we write the spherical function, i.e., 

*fc(g) = fa(g)£,0- 

By (13.11 $h is a function on K (0) \ G/if(0), i.e., on the set of non-labeled 
checker-boards. 

Let us write natural numbers i dim U on red edges of the simplicial com- 
plex, j ^ dimV^ on yellow edges, and k ^ dimVy on blue edges. For each 
triangle T, we get 3 numbers on its sides, say i(T), j(T), k(T). 

Proposition 4.2 

#h(a) = 

H II hi(T)j{T)k(T) ]J hi(S)j(S)k(S) 

,j j. r black trianqles T white triangles S 

all arrangements oj 3 ^ 

i, j, k on edges 

(4.1) 

Proof. We choose a representative g € G of the coset a such that gi, g2, gz 
arc finite, let they are contained in Sn . Hence we can work in the space H® N , 

h ®N = 

( EI h i m ,j m ,kS) («•! 8^0^)0-^ (it ijv <S> Uj w ® w kN ) (4.2) 

Thus summands are enumerated by collections of numbers written on sides of 
black triangles. 
Next, 

p(g)h^ = J2 

iiit2i...; jiij2i."i fci,fc 2 ,... 



( LI J« > fc ™ ) Kl (ii ) ® W 92 ( ji ) ® ^93 (fci ) ) ® ■ • ' ® Kl (ijv ) ® W 92 ( JJV ) ® W 93 (fc„ ) ) 

(4.3) 

We change the order of summation, 
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and get 

E 

M2,---, ji-j2;---, kiifa,... 

m<iV 

Evaluating the inner product of (|4.2p and (|4.3[) . we come to (|4.1[) . □ 

Remark, a) The multiplication in -ftT(O) \ G/K(Q) is the disjoint union. 
Therefore, for a, fa £ K(0) \ G/K{0) we have 

<P h (ao fa) = $ h (o)$ h (b). 

Factors corresponding to chebureks are 1. 

b)The function a i— * $ft(o)$fc/(a) has the form $^(o). Indeed, having spaces 

H=(U®V®W 1 h) and H' = {U' ®V ®W , h') , 

we take the space 

H®H' ={{U® U') ®{V® V) ®{W ® W'), h®h'Y 

4.3. Representations of the category §. Now we wish to construct the 
corresponding representations of the category of checker-boards. 

Proposition 4.3 The spaces of K (a) -fixed vectors are 

H(a) = H ® ■ ■ ■ ®H ®h ®h® 

a times 

This a rephrasing of Proposition 14. II 

In particular, H(a) are finite dimensional iff U, V, W are finite dimensional. 

Now let us write matrix elements for a checker-board a <E K(a) \ G/K(f]). 
Consider basis vectors 

£ := (it*! ® v n ® Wk 1 ) ® ■ ■ ■ <8> (tif p ® Wj^ ® Wkp ) ® h® h® . . . G H(/3), 

£' := (u^ ® Uj/ ® w^) ® ■ ■ ■ ® (ui' a ® vj' a ® Wk> a ) ® h® h® ■ ■ ■ € H(a). 

Then we get the same formula (|4.1[) . but a set of summation changes. Now we 
are obliged to write (i s ,j s , k s ) (according colours) on edges of the white triangle 
with labels s ^ j3 and (i'^,^, k' a ) on the black triangle with label a ^ 7. The 
products are given over non-labeled triangles. 

4.4. Super-tensor products. The previous construction admits an ex- 
tension. Let each Hilbert space U, V, W be Z2-graded, 

U=Uo®U r , V = Vo®V r , W = W S ®W T , 
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sums are orthogonal. 

Take bases Ui £ U, Vj £ V, Wk £ W compatible with gradations (i.e., 
ui £ Uq or Uj). Take an even vector 

heH =:U ®V ®W, 

i.e, 

h £ (Uq O ® W%) © (E% © Vy © Wj) © (t/ r ® Vfr ® PF T ) © (U r <8> Vj © Wq) 

Next, we consider the tensor product 

H := (H, h)®(H,h)®... (4.4) 

For Z2-graded spaces Y = Yq © Y\, Z = Zq ffi Z\, the operator of transposition 
of factors 

Y ®Z ^ Z®Y 

is given by 

Vo ® % ^ % ® 2/o ^0 ® Z T ^ Z T ® 2/o' 

J/1 © Zq i-v z 7 © y-, yj ® z T ^ -z T © 

This determines the action of symmetric group in tensor products of Z2-graded 
spaces. 

Taking the action of G in the infinite tensor product (|4.41 we get a unitary 
representation of G. 
SS 

4.5. Spherical functions in super-case. We get the following expression 
similar to P~Tj) : 

$h(a) = 

= x^-ir n h i(T)j{T)k{T) n h i{s)j{s)k{s) (4.5) 

black triangles T white triangles 5 1 

We must describe a set of summation and a choose of signs (— 1)°\ 

We write basic vectors on edges of the checkers-board according the following 
rules: 

1) Ui on red edges, Vj on yellow edges, Wj on blue edges. 

2) The perimeter of a triangle contains even number of odd basis elements, 
i.e., or 2. 

Thus, consider a triangle T having 2 odd basis elements on its perimeter (we 
say 'odd edges'). Then its neighbors through odd edges also have 2 odd edges. 
Consider their neighbors through odd edges etc. In this way, we get a collection 
of closed chains of triangles. Denote by 21 1 their lengths (these numbers are 
even). Then 

cr = $>-l). 

t 
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Figure 4: A gallery of triangles with two odd edges 

5 n-symmetric group 

Here the situation is similar to the 3-symmetric group. All consideration given 
above (checker-boards, the multiplication of double cosets, Theorem 13.11 and 
constructions of representations) survive for general n ^ 2. 

However, there is an alternative (below 'quasidual') description of spaces of 
double cosets 

K{a)\<G [n] /K{P). 

For n > 3 the two descriptions are essentially different. 
SS 

5.1. Polygonal checker-boards. We write an element p of G„ as p = 
(Pi, ■ ■ ■ ,Pn)- 

Consider a countable collection A±, A%, ...of oriented black n-gons. On 
sides of each Ak we write anticlockwise numbers 1, 2, . . . , n. Consider another 
countable collection of oriented white n-gons B%, B2, .... On sides of B\, 
Bit ■ ■ we write clockwise numbers 1, 2, . . . , n. 

For each i ^ n for each m S N we glue i-th side of A m with z-th side of 
Bpi(m)- ln this way we get an n-gonal complex, which is a union of countable 
collection of compact closed two-dimensional surfaces. 

Again, almost all components are chebureks, i.e., obtained by gluing of a 
black and white n-gons along their perimeters. 

Now we can repeat one-to-one all considerations of Sections 2-4. 

5.2. The quasi-dual polygonal complex. We draw a countable collec- 
tion of black vertices a\ , ai , ... and a countable collection of white vertices b± , 
62, .... For each i ^ n for each m 6 N we draw an edge [ai, b Pi ( m )] an d draw 
the label i on this edge. In this way, we get a graprH. 

Next, for any i, j ^ n, we consider the permutation p~ 1 pj. Decompose it in 
independent cycles. We get chains of the form 

Pi . Pi 1 Pi , Pi 1 Pi Pi , Pi 1 
Sl I— ► t\ I ► S2 I— ► 12 > S3 1 — »■ • • • I — >• Cfc I— > Si 

e Which is dual to the 1-skeleton of the checker-board. 
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Figure 5: A cheburek and a quasidual cheburek for n — 4. 

a) In checker-board model a cheburek is obtained by gluing of two quadrangles 
along their perimeters. 

number of faces = 2, number of vertices = 4, number of edges = 4. 

b) In the quasidual model, we glue 6 biangles: 

number of faces = 6, number of vertices = 2, number of edges = 4. 




Figure 6: n = 3. A checker-board and the dual complex. 
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Figure 7: Gluing of quasidual polygonal complexes. Picture at a vertex. 
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Then sitiS2^2 ■ ■ ■ tk s i is a closed way on our graph. We glue a 2Z-gon to this 
way. Thus we get a polygonal complex. 

We get n(n — l)/2 polygons meeting in each vertex. Each pair of polygons 
has a common edge. Therefore, for n > 3 our complex is not a 2 -dimensional 
surface. See Figure O 

Remark. For n = 3 these constructions are dual in the usual sense, see 
Figure [H □ 

To pass to double cosets K(a) \ G^ /K{j3), we remove labels > (3 from black 
vertices and labels > a from white vertices. 

It remains to describe the multiplication 

K (a) \ G^/K{[3) x K{0) \ G^/K{j) K(a) \ G^/K(j) 

For each i ^ /3 we take the link of the black vertex bi £ K{j3)\G^ / K{~{) and the 
link the white vertex at G K (a) \ / K (0) . We remove neighborhoods of both 
the vertices and get collection of half-edges in each complex. So we glue halves 
of edges with the same labels together. We must describe two-dimensional faces 
of the new complex. We glue the corresponding polygons as it is shown on 
Figure [71 



6 A priori theorems about representations of n- 
symmetric groups 

Here we use the checker-board model for double cosets. 

6.1. Representations of products of group. Let G\, G% be finite 
groups. Then any irreducible representation of G\ x G*2 is a tensor product 
Ti xr 2 of irreducible representations of G\ and G 2 . A similar theorem for unitary 
representations holds for compact groups, semisimple groups (over K. orp-adics), 
nilpotent Lie groups. However it is not the case for unitary representations of 
arbitrary groups. Apparently, a minimal necessary condition (see, e.g., [1]) is: 

— the group G\ has type I. 

We can restrict an irreducible representation of G^ to the dense subgroup 
S^ n x ■ ■ • x S-£ n . But it is not a tensor product of representations of S^ n . 
Moreover, consider two classes of representation of S[^ n x • • • x S^™: 

— the set of tensor products n <8> • • • <8> T n ; 

— the set of representations admitting a continuous extension to the group 

GH 

The intersection of these two huge classes is almost trivial. Precisely: 

Proposition 6.1 a) Each irreducible representation t% • ■ • ®r n of S^ n x • • • x 

S^ n having a non-zero K-fixed vector is one-dimensional. 



1G 



b) Let pj be irreducible representations of S[^ n . The representation t\ ® 
■ ■ ■ ®T n admits a continuation to G^ if and only if each Tj is continuous in the 
topology of Soo ■ 

Proof is given in Subsection 16.31 
SS 

6.2. Reformulations of continuity. 

Theorem 6.2 a) Let an irreducible unitary representation p of S^ n x • • • x S[^ n 
have a non-zero vector invariant with respect to the diagonal subgroup K . Then 
p is continuous in the topology of G*- n ' . 

b) Let an irreducible unitary representation p of S(^ n x • • • x S^ n have a 
non-zero vector invariant with respect to some subgroup K{a) C K . Then p is 
continuous in the topology of G^ . 

c) Let t be a unitary (reducible) representation of S££ n x • • • x S£™ in a 
Hilbert space H. Denote by H(a) the space of all K(a)-fixed vectors in H. 
Then the following conditions are equivalent: 

— t is continuous in the topology of G^ . 

— the space Off (a) is dense in H . 

Olshanski's |13j proof for n = 2 survives for any finite n. 

We formulate separately this criterion for the group = Sq^. 

Theorem 6.3 Let t be a unitary (reducible) representation of SQ n in a Hilbert 
space H . Then the following conditions are equivalent: 

— t is continuous in the topology of Soa . 

— the space UH(a) is dense in H . 

If t is irreducible, then these conditions are equivalent to 

— the space L)H(a) is non-empty. 

6.3. Proof of Proposition 16.11 

Lemma 6.4 Let T\, be unitary (generally, reducible) representations of a 
group G. Assume that T2 has not finite- dimensional G-invariant subspaces. 
Then the representation t\ ®t% also has not finite- dimensional G-invariant sub- 
spaces. 

Proof. Denote by V\, V% the spaces of the representations t±, r%. Assume 
that V\ ® Vi admits a finite-dimensional invariant subspace W, denote by ip the 
subrepresentation in W. 

Then the tensor product ip* ® T i ® ti admits a G-invariant vector r\. Indeed, 

W* ® (Vi O Vi) D W* i® W. 

We can regard as the space of operators W —> W, the identical operator 

commutes with the action of G. 
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Next, we identify W* <g> V\ <g> V<i with the space of Hilbert-Schmidt operators 



W®Vi -> V 2 . 

An invariant vector r\ corresponds to an intertwining operator W ®> V^* — > V%. 
We choose bases in the initial space W ® Vj* and in the target space V2 such 
that the matrix of A is diagonal. Equivalently, we expand A in a series 



where Xj are the singular numbers of A, and Rj are partial isometries, 

imi?i_Limi?j, imi?*±imi?*. 

Then the operators are intertwining, hence im Rj is an invariant subspace. 
However rk(i?j) is the multiplicity of the singular number Xj, it is finite. There- 
fore imi?j =0 and Ri = 0. □ 

Proof of Proposition 16.11 The group S^ n has two one-dimensional 
characters, the trivial one and the signature. They are the only irreducible 
finite dimensional representations of S[^ n . Note also that the signature does 
not admit an extension to the complete group Sao. 

Let a representation n <8> • • • <g> r„ of S^ n x • • ■ x S^ n has a K(a) ~ S£ n (a)- 
invariant vector. By Lemma l6.4l each Tj has an S'^"(a)-eigenvector «j, 

T j(9) v j = Xj(g)vj, geS^ n (a), Xj is a character of S£ n . 

Therefore all representations Tj ~ Xj T j have S 1 ^™ (a)-invariant vectors and are 
continuous in the topology of Soo- 

Thus, our representation of G^ has the form 

(gi,...,g n ) i-> (]J Xi(5j))n(ffi) ® ••• ®?„(g„). 
Since the representations 

(51,.. .,g n ) i-> r( a ) 

are continuous in the topology of G™, it follows that the character IlXj(5i) 
also is continuous. Therefore it is trivial. □ 

6.4. Analogy with p-adic groups. The criterion of continuity (Theorem 
16. 2p is an imitation of admissibility for p-adic groups. For definiteness, consider 
G = GL(n, Qp). Denote by GL a the subgroup consisting of matrices 1 +p a T, 
where T has integer coefficients. Then a representation p of G in a space H is 
admissible if each vector is fixed by some subgroup GL". A unitary representa- 
tion of G in a Hilbert space H is admissible, more precisely it is admissible on 
a dense subspace. 

The reason of this parallel, see [5], Proposition VIII. 1.3. 
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6.5. Correspondence between representations of the group 
and representations of the category S^ n '. In Section 3 we obtained the 
canonical map 



Unitary representations of G'"' > — > \ ^-representations of S 




Theorem 6.5 This map is a bijection. 

Below we prove a stronger version of this statement. 

6.6. The completion of the category §I n l. Let us define a 'new' category 

— [n] 

S obtained by an adding of an infinite object. Its objects are 0, 1, 2, ... , oo. 
A morphism j3 — > a is the following collection of data: 

— a finite or countable collection of checker-boards colored in the usual way; 

— almost all components are chebureks; 

— we fix an injective map from the set {1,2,..., /3} to the set of black faces 
and an injective map {1,2,..., a} to the set of white faces; 

— there is no empty chebureks. 

The product of morphisms is given in the usual way. 

We use the common notation Mor(/3, a) for sets of morphisms (3 — > a, End(a) 
for semigroups of endomorphism, and Aut(a) for group of automorphisms. 
Evidently, 

Aut(a) = S a X ••• x S a , where a < oo, Aut(oo) = G'"l 

At the end of this subsection we define the natural topology on the semigroup 
End(oo) (representations must be continuou^3) and discuss some properties of 
the category S . Now we formulate the main statement of the subsection. 

Theorem 6.6 Any * -representation of the category §[™1 admits a unique con- 

— [n] 

tinuous extension to the * -representation of the category S 
We need the following distinguished morphisms 



defined for all a > (3. 

The morphism \p, a : (3 — > a is defined as a disjoint sum of chebureks with 
the following labels on black/white sides: 



A/3 :Q : f3 —y a, 



Va,p ■ a — » f3, 




a 



(j/j), where j < (3, 
(p/empty), where (3 < p $J a. 



We set 




73, a> 




7 With respect to the weak topology on the space of operators in a Hilbert space 
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Thus Ha,/3 is determined by the following collection of chebureks: 



(j/j), where j < (3, 
(empty /p), where (3 < p ^ a. 



and 02 by 



(j/j), where j < /3, 
(p/empty), (empty /p), where /3 < p ^ a 



(6.1) 
(6.2) 



We need some simple fact concerning these morphisms. 
First, 




(6.3) 
(6.4) 



Second, the map 



(6.5) 



is an injective homomorphism of the semigroup End(/3) to the semigroup End(a). 
In fact, we add chebureks (|6.2p to the checker-board a. 
Third, we need the map End(a) — > End(/3) given by 



This is equivalent to removing of labels > j3 from black and white triangles. 

Now we are ready to define the topology on End(oo). We say that a sequence 
Cj G End(oo) converges to c if for each (3 < oo 

fJ-oo,/3CjXp,oc = fJ-oojjcXf}^, for sufficiently large j. 

Example. A weak zero hjK(a), see Proposition ^. 61 converges to 9^. □ 

Proposition 6.7 a) The multiplication in End(oo) is separately continuous, 
b) The group Aut(oo) is dense in the semigroup End(oo). 

Proof of b. Let c e End(oo) have non-labeled white and non-labeled black 
faces. For each f3 let us make dp G End(oo), where all black faces are labeled, 
in the following way. 

We preserve all white labels on c, preserve all black label ^ (3, remove black 
labels with numbers > (3, and enumerate all initially non-labeled black faces 
and all faces liberated from labels in arbitrary way. Then dp converges to c. 

Next, for each dp we construct a sequence ep a by repeating the same actions 
for white faces. Then tp a converges to dp. 

On the other hand, tp a 6 End(oo). □ 

6.7. A formal proof of Theorem 16.61 It is a special case of Theo- 
rem VIII. 1.10 from [5]- All necessary structures were defined in the previous 
subsection. We must only verify: 



a 
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Lemma 6.8 For any * -representation p of the category for any morphism 

llp(a)|Kl. 

Proof. By (|6.3p - (|6.4p . p(0g) is an orthogonal projection and p(Xj3, a ) is an 
isometric embedding. For c S Aut(7), the operator p(c) is unitary. 
We can represent any a £ Mor(/3, a) as 

o = p, lta ci/p tl , c e Aut(7) 

for some 7. We simply write labels on all empty triangles. This completes the 
proof. □□ 

6.8. Reconstruction of a representation of the group G^ from a 
representation of the category. We will not repeat the proof of Theorem 
VIII. 1.10 from [5]. Here we briefly describe the limit pass necessary for Theorem 



Let p be a ^-representation of . Let H(a) be the spaces of representation. 
For (3 < a, the operator p(Xp, a ) is an isometric embedding H{[3) — > H(a). 
The semigroup End(/?) acts in the both spaces, see (|6.5P ; the operator p(\p ta ) 
intertwines these actions. 

Therefore, we can assume that H((3) C H(a) and take the inductive limit 
H(oo) of the chain 

•••->#(/?) -!T(/3 + l)-... 

of Hilbert spaces (i.e., we take the completion of UpH(/3)). The inductive limit 
U^<ooEnd(/3) of the chain of semigroups 

► End(/3) -» End(/3 + 1) -> . . . 

acts in H(oo). 

However, the semigroup U ) g< 00 End(/3) is a small subset in End(co) (U ( 3< 0O End( / 3) 
is countable and contains no invertible elements at all). To define p(c) for an 
arbitrary c G End(oo), we write p(c) as a weak limit 

p(c) := lim THfifap). (6.6) 

6.9. The composition of functors. Thus we get a chain of 3 functors: 

r Unitary \ Tl / *-Representations'i t 2 

\ representations of G^ J I of / 

^-Representations i Ta r Unitary "i . . 

'l ofS 1 " 1 J ' I representations of G [ "U ^ ' 

The last map is the restriction of a representation of End(oo) to Aut(oo) ~ G^. 
We must show that this chain is closed, 

Proposition 6.9 The composition T3T2T1 is the identical map. 
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Proof. Let p be a unitary representation of G^. Let b € End(/3), where 
(3 < oo. In Section 3, we defined the operator 75(b) as follows. Take g^ £ G^ 
such that 

b = eft P 6f. (6.8) 

Then 

/5(b) := P(fl)p(gp)P(p). (6.9) 
Take g £ Aut(oo) ~ G^. Keeping in the mind (|6.6[) - (|6.9p , we get 

p(g) = lim P(/?)p(g)P(/3) (6.10) 

p — >oo 

(the limit is a weak limit). By Theorem l6.21 P{(3) strongly tends to the identical 
operator. Therefore the weak limit (|6.10p is p(g). 

Thus, 75(g) = p(g) for g e GN. □ 

Next, we prove a stronger form of Proposition 16.91 

Theorem 6.10 All the mapsTi, T 2 , T 3 are bijections. The composition T 3 T 2 Ti, 
T2T1T3, T1T3T2 are identical. 

Proof. The injectivity ofT\. This follows from the identity 

lim P(f3)p(g)P(p) = p(g), 

see the previous proof. 

The bijectivity ofT 2 is evident, the map T 2 _1 is the restriction of a represen- 
tation of E [n] to sH 

The injectivity 0/T3. The group Aut(oo) is dense in the semigroup End(oo). 
Therefore a representation of Aut(oo) remembers a representation of End(oo). 
On the other hand, all the semigroups End(a) can be regarded as subsemigroups 
in End(oo). □ 

6.10. Uniqueness of a spherical vector. 

Theorem 6.11 Let p be a unitary irreducible representation of G^, let H(0) 
be the space of K -fixed vectors. Then 

dimff(O) < 1. 

Proof. Our functors (|6.7p send direct sums to direct sums and irreducible 
representations to irreduciblq^ representations. In particular, the representation 
of End(O) ~ K(0)\&W/K(0) must be irreducible (see, e.g., [5], Lemma II.8.1.). 
The semigroup ~ K{0)\G^ /K(0) is a commutative semigroup with involution. 
Its irreducible ^-representations are one-dimensional. □ 

If dim_ff(0) = 1, we say that the representation is K -spherical. 

6.11. Spherical characters. For a — 0, 1, . . . , 00, denote by A(a) the 
center of End (00). 



'For categories, see definitions in [5], II. 8. 
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Proposition 6.12 All elements o/A(a) have the form: the collection of labeled 
chebureks 

U/j), where j < a, 
and a finite disjoint union of compact non-labeled surfaces. 

The statement is obvious. 

Thus, all semigroups A (a) are isomorphic to 

A:= A(0) = J ftT(0)\G™ 1 /^(0). 

For a > (3, the isomorphism A (a) — > A(/3) can be written as follows: 

3 l-> Ma,/33^/3,a- (6-11) 

The center of End(oo) acts in irreducible representations by scalar operator. 

Observation 6.13 For any irreducible representation p of G^, we get a ho- 

momorphism Xp of A to the multiplicative semigroup of C. 

We say that Xp is the spherical character of the representation p. 

Observation 6.14 For any irreducible spherical representation ofG^ the spher- 
ical character coincide with spherical function. 

Lemma 6.15 For any irreducible representation p of G^ for each a the semi- 
group A(a) acts in H{a) as Xp- 

PROOF. We apply the map (|6.1ip . □ 
6.12. Self-similarity and spherical characters. Consider the subgroup 

G*- n '(a) C G^ that fixes points of the initial segment 1, . . . , a in all n copies 

ofN. Then 

G w (a) ~ G [n] . 

Restricting a representation of G^ to G^(a), we get a collection of (reducible) 
representations of G'™'. 

Evidently, the embedding 

G w (a) -^G w 

induces an isomorphism of centers of two copies of End(oo). This implies the 
following statement: 

Theorem 6.16 Let p be an irreducible unitary representation o/Gl™'. Consider 
the restriction 

p 

GW(a) 

and a spherical subrepresentation t of the restrictioi^. Then the spherical func- 
tion of t coincides with the spherical character of p. 



'For sufficiently large a such subrepresentations exist. 
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1 10 2 5 4 
4 ^' ■ - • ■ - • "\ 9 





7 2 8 6 5 

Figure 8: Example. A pair of substitutions 



o 



10 



1 2 4358769 10 
4 10 5936281 7 



Notation for colors: 
black 

white 



red 
blue 



1 24358769 10 
10 59362847 1 



Example. A representation of & n ' is continuous in the topology of Soo x 
• • ■ x £00 (see Subsection 6.1) if and only if its spherical character = 1 (this is a 
rephrasing of Proposition 16. ip . □ 

Let x be a spherical character. For each connected non-labeled checker-board 
c we have a number x( c )- F° r a disjoint sum Ci U • ■ ■ U c m we have 



x(ciU-.-Uc m ) = n^( c i)- 



Thus a spherical character is determined by numbers x( c )> where c ranges in set 
of connected checker-boards. 

However, numbers x(c) are not arbitrary. 

6.13. Status of problem of classification of unitary representations 
of the group G™. It is not clear, is this problem reasonable or not (there are 
different arguments 'pro and con'). 

However, the problem looks like two-step: 

a) to describe all possible spherical characterful: 

b) to classify all irreducible representations with a given spherical character. 
There is also (apparently, simple) problem: 

c) to prove that G^ is a type I group. 

For the bisymmetric group G^l, the description of spherical characters is 
equivalent to the famous Thoma Theorem (1964), see Theorem lA.il The prob- 
lem b) was solved by Olshanski [T3] and Okounkov [9]. 



10 The construction of Section 4 does not exhaust all spherical characters. Even for Gt 2 ' 
there is an additional parameter. 
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Figure 9: The analog of a chcburek 



Figure 10: The production of a chip from an one-dimensional simplicial complex. 
We take an element of K{4) \ G^/K(3): 




4' 3' 2' 1' 1" 2" 3" 4' 
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A. Addendum. Bisymmetric group 

A.l. Olshanski chips. Let n — 2. Consider an element g — (g r ed, guue) G 
G' 2 '. Take a countable collection of black segments Aj with red and blue ends. 
Take a countable collection of white segments with red and blue ends. If g re d i = 
j, then we glue the red end of A4 with the red end of Bj. If guuek = I, then 
we glue the blue end of Ak with the blue end of £?/. In this way, we get an 
one-dimensional complex. One-dimensional strata of the complex (segments) 
are colored in black and white, zero-dimensional strata in red and blue. There 
are labels 6 N on black segments and labels G N on white segments. 

The pair g = (g re d, guue) can be easily reconstructed from these data, see 
Figure [5J 

To realize a double coset 

eK(a)\&W/K(J3), 

we remove labels > from black segments and labels > a from white segments. 
The analog of an empty cheburek is a circle consisting of two non-labeled seg- 
ments. It is reasonable to throw out all empty chebureks (see Figure 0. 

Thus we get a finite disjoint union of closed chains of segments, black and 
white segments interlace, red and blue ends also interlace. There are pairwise 
different labels 1, ...,/? on some black segments and pairwise different labels 1, 
. . . , a on some white segments. 

Next, it is reasonable to do the following operation: 

1. We cut all labeled segments at midpoints and assign the corresponding 
labels to the ends. 

2. After this we get a finite collection of cycles having even lengths 2l p and 
a finite collection of non-closed chains. If both ends of a chain are white (resp. 
black), then the chain contains an odd number 2m q + 1 of complete segments. If 
one end is black and another is white, then the chain contains an even number 
2k r of complete segments. See Figure [POl 

Now an element of K(a) \ G^ /K(/3) corresponds to diagram (a chip) of the 
following form (see Figure [TU]) : 

a) There are /3 + (3 entries. On Figure we show them by f3 labeled fat points 
and P labeled square points. There are a + a exits. On Figure [TOl we show them 
by numbered a fat points and numbered a square points. 

b) There are arcs of 3 types (horizontal, vertical, and cyclic). For each arc 
we assign a non- negative integer (length). The rules of designing are: 

— a horizontal arc connects a square point and a fat point in the entrance 
or a square and a fat point in the exit. It has odd length. 

— a vertical arc connects a fat point in the entrance with a fat point in the 
exit or a square point in the entrance with a square point in the exit. It has 
even length. 

— cycles have even length. 
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Figure 11: A bianglular checker-board. We omit colors on edges. 



Having two chips a : 7 — > (3 and b : j3 — ► a. Then we connect exits of a with 
corresponding entrances of b and evaluate lengths of composite arcs. 

Remark. Certainly, we can apply the general construction of checker-board 
for G' 2 '. In this case 2-dimensional complexes are spheres divided into biangles, 
see Figure [TTJ 

A. 2. Characters of Soo and the bisymmetric group. In 1964 E. Thoma 
obtained the classification of all characters of S^ n . According his definition, a 
character is an extreme point of the set of all functions F on S^ n satisfying the 
conditions: 

— F is central, i.e., F(hgh~ l ) = F{g)\ 

— F is positive definite; 
-F{l) = l. 

Theorem A.l (Thoma, |14|) All characters of S[^ n have the form 

^)=n(E«M(-ft) t ) rtW (A.i) 

fc^2 j j 

where rk{g) is the number of cycles of length k in g, and the parameters a, [3 
satisfy 

a 1 ^a 2 ^...^0, Pi>fo>...>0, X) Q y + E#»'^ L 

Having a positive definite function \ on t ne group S[^ n , we define a Hilbert 
space and a unitary representation of the group in the usual way, see, e. g., [T]. 
This representation is a //i-factor representation (in particular, it is strongly 
reducible). 

Theorem A. 2 flfflj). a) There is a canonical one-to-one correspondence be- 
tween Thoma characters and spherical representations of the bisymmetric group. 
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b) Moreover, for a character xid) the corresponding spherical function is 
given by 

$(91,92) = xigig^ 1 )- 

Formula (A.l) is simpler than our formulas (|4.ip and (|4.5p . In fact a vector 
h € U <E> V, where U, V are Hilbert spaces, 
can be reduced to the form 

by a change of bases in {/ and V. For a triple tensor product such reduction is 
impossible. 

A. 3. Symmetric pairs. Olshanski researches in infinite-dimensional groups 
(see [T2], [13]) partially are an imitation of symmetric pairs and the theory of 
spherical functions. In Neretin [5], VIII. 5 it was observed that some tricks 
related to infinite dimensional symmetric pairs can be applied in a wider gen- 
erality. 
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